1. Technical Field
This invention relates to an image processing apparatus and the like used for a digital camera or the like having a color image pickup sensor such as a CCD or CMOS, and more particularly to an image processing method, an image processing apparatus, an image processing program, and a semiconductor device that are aimed at processing to reduce or eliminate a false color generated by color interpolation processing.
2. Related Art
Generally, color cameras for inputting color images include a three-plate type camera using three image pickup sensors such as a CCD (charge coupled device) or CMOS (complementary metal oxide semiconductor), a single-plate type camera using only one such image pickup sensor, and a two-plate type camera using two such image pickup sensors. The three-plate type color camera is adapted for performing color separation of incident light into the primary colors of red (R), green (G) and blue (B) by dichroic mirrors or the like and converting light information of the separated colors into digital images of red, green and blue by three image pickup sensors. Since the three-plate type color camera can acquire high-quality images, though its optical system has a complicated structure, it is used mainly as a color video camera for professional use, a high-grade digital still camera or the like.
On the other hand, the single-plate type color camera has color filters of red, green and blue arrayed in a block-like manner on one image pickup sensor so as to digitize light information of each pixel transmitted through the corresponding color filters. Since the single-plate type color camera has small size and light weight, consumes less power and is available at a low price, though its image quality is slightly lower than that of the three-plate type color camera, it is mainly used as a private color video camera or digital still camera.
By the way, some private color video cameras employing the three-plate system have been recently commercialized. The two-plate system, which is a system having intermediate characteristics relative to the other two systems, is not used very often at present.
In the single-plate type color camera as described above, because of its structure, color interpolation processing to predictively interpolate a color component that is missing is required. However, this color interpolation processing generates data of a color that does not exist at a target pixel by estimation from pixel values around the target pixel, as will be later described in detail. Therefore, in many cases, an accurate value cannot be calculated at a part where the pixel value changes largely, such as an edge part. Accordingly, a pixel having color balance that is different from the original color balance is generated at the edge part where an accurate value cannot be calculated, and this appears as a false color, which is one of the factors lowering the image quality.
As a technique for reducing such a false color, for example, a technique of inserting an optical low-pass filter (hereinafter simply referred to as “LPF” on suitable occasions) or the like before the image pickup sensor to reduce the false color may be considered. However, this has a disadvantage in that a high spatial frequency component (hereinafter simply referred to as “high-frequency component) is cut by the low-pass filter effect, thus lowering the sharpness or MTF (modulation transfer function or resolution) of the image. Moreover, in this technique, it is difficult to eliminate a false color generated by image processing on the subsequent stage, and to accomplish this purpose, a frequency band ranging to a significantly low frequency component must be cut.
Thus, conventionally, (1) the saturation at the edge part where a false color tends to occur is lowered to make the false color less visible, as disclosed in JP-A-2003-102025, JP-A-11-308625 or the like, and (2) color information is judged more accurately and a color signal that does not generate a false color is generated to restrain the occurrence of the false color, as disclosed in JP-A-2001-211459 or the like. Also, techniques such as (3) calculating peripheral information and selecting a combination of colors that minimizes the saturation so as to make a false color less visible, as disclosed in JP-A-2002-77930 or the like, and (4) calculating a median of color signals including color signals of peripheral parts and eliminating a color that is considered to be a false color so as to reduce the false color, as disclosed in Japanese Patent No. 2,612,850 or the like, have been proposed.
Techniques (1) and (3) of the above-described conventional techniques involve processing to lower the saturation. Since an edge must be determined and the processing is directed to reducing the saturation, the processing is complicated and it is probable that correct information cannot be acquired. That is, data with no color may occur on the periphery of an edge, and in some cases, the original color is not reproduced and may appear like a false color (false contour).
On the other hand, technique (2) can exhibit an excellent false color avoidance effect, but this technique is inconvenient in that very heavy processing such as division for each pixel is required.
Technique (4) is simple because of the idea of processing the value of the difference of color-interpolated data in the target area through the median filter. However, if the processing target area is made two-dimensional or the like and the number of processing target pixels increases, it is probable that the processing becomes heavy.
Another simple technique for avoiding a false color is to process a color component that generates a false color through an LPF and thus disperse the false color generated at an edge part to its periphery so as to reduce the absolute magnitude of the false color and make the false color less visible, as will be described later. However, this technique only disperses the false color component to the periphery and cannot fundamentally eliminate the false color.
Thus, the present invention aims to effectively solve such problems. It is an object of this invention to provide a new image processing method, image processing apparatus, image processing program and semiconductor device that enable easy and effective reduction or elimination of a false color generated at an edge part because of color interpolation processing.